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MULTI-MEDIA INFORMATION RECORD 
DEVICE, AND A MULTI-MEDIA 
INFORMATION PLAYBACK DEVICE 

BACKGROUND OF THE INVENTION 

(1) Field of the Invention 5 
The present invention relates to a multi-media informa- 
tion record device and a multi-media information playback 
device: the muiti -media information record device records 
multi-media information like dynamic /still picture, 
graphics, sound and the like in a storage device and a 10 
recording medium provided to a data server: the multi-media 
information playback device plays back the multi-media 
information inputted via a recording medium and a trans- 
mission line. 

(2) Description of the Related An 15 
Recently. TV game. sets, electronic publications, quality 

karaoke sets and video on demand devices are used as 
read-only multi-media information playback devices. These 
multimedia information playback devices play back multi- 20 
media information recorded in a optical disc (CD-ROM) and 
a digital video disc (DVD), and also multi-media informa- 
tion transmitted from a data server via a network and the 
like. 

The following explanation is about "Denshi Karaishibai- 15 
Electronic Picture-Card Show" disclosed in Japanese Laid- 
open Patent Application No. 5-28648. which is an example 
of the conventional multi-media information playback 
device mentioned above. FIG. 1 is an example of a data 
configuration of multi-media information of a conventional ^ 
multi-media information playback device, by which images 
and sound are synchronously played back. 

Data and information shown in a data area section and 
control information section of FIG. 1 are recorded in the 
recording medium like a digital disc. Control information 35 
shown in the control information section is recorded in each 
block composed of predetermined amount of data shown in 
the data area section, in which sound data and still picture 
data mingle. In the data area section of FIG. 1. An (n is a 
natural number) represents sound data: VnB and VnC rep- 40 
resent still pictures A and B. respectively; CLUT (color look 
up table) represents CLUT data; NOPn represents data to 
which no operation is applied. In the control information 
section. (1) represents a data write command; (2) represents 
a CLUT data command: (3; represents a data display com- 45 j 
mand: (4) represents an audio data command; (5) represents 
a command of no operation. In the screen image section. A 
and B represent still pictures A and B. respectively. Both of 
them are displayed on the screen. ; 

Each command in the control information section is 50 ! 
executed in order from the left side to the right side. First, i 
audio data Al, which is digitalized sound information, is |. 
transferred to audio data memory by the control information ' 
(4). and outputted as an analog audio signal via an audio D/A 
(digital/analog) converter. Video data V1B is written in a 55 
predetermined position of video memory of V42B from 
V1B. after the write area of a still picture is designated by 
the control information (1). The CLLT data updates the 
screen image by the control information (2). Based on a 
CLUT memory, contents of the video memory are converted 60 
to an analog video signal by the video D/A converter, so that 
the contents are displayed on the screen. While the screen 
image and an audio signal A44 in FIG. 1 are synchronized, 
a still picture A can be switched to a still picture B. Although 
the succeeding still pictures C D. E ... are not shown in 65 
FIG. 1. the same processing is carried out to play back 
display screens and sound synchronously. 
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The next explanation is about a general playback process- 
ing of a multi-media information playback device. Multi- 
media information playback device places a line end signal 
and a page end signal, which are already inserted in the 

5 digitalized image signals, at the end of a scanning line 
(normally more than 500) and a page (contents of a screen 
at every Vx> second) in the display device like Cathode Ray 
Tube and Liquid Crystal Display, respectively, and displays 
the scanning lines on the proper pages, so that image signals 

10 recorded in the recording medium like a CD-ROM and a 
DVD axe displayed at predetermined positions on the screen 
of the display device. M u It i- media information playback 
device quadruples a size of an image to be displayed on the 
screen by using two pixels vertically and horizontally, which 

15 forms the smallest square of four pixels, for each pixel in the 
display device, or by interpolation. This device also reduces 
an image to a quarter size, by skipping a pixel vertically and 
horizontally. 

Also the mu ld-media information playback device rotates 
20 an image to be displayed on the screen by converting 
coordinates of each pixel using trigonometric function. As 
pixels are more dense in the recording medium than in the 
display device, predetermined adjustment is applied to the 
recording medium so that a proper image is displayed based 
25 on the pixel density of the display device, in time of normal 
display. So practically, there's no problem when an image 
which is stored in the CD-ROM as a small still picture is 
enlarged as much as possible on the screen of the display 
unit. 

30 Generally, the multi- media information playback device 
is equipped with necessary programs to play back the 
multi-media information stored in the recording medium- 
Some times, a different program is already written in the 
multi-media information so that commands which are added 

35 to the multi-media information or which are changed are 
executed according to the program. In this case, signs which 
are similar to T\ and are given to the different program 
which is described by the multi-media information, so that 
the multimedia information playback device recognizes the 

40 program to be executed. 

Moreover, the multi-media information playback device 
displays a still picture, which does not change at all during 
the display time, in the display device by repeating the steps 

45 of reading and displaying the display data stored in a VRAM 
(video RAM) every Vx> second or so. On the other hand, 
dynamic pictures are displayed by repeating the steps of 
updating the display data stored in the VRAM, reading and 
displaying the updated display data every Vy> second or so. 
Therefore, one storage unit of page data which shows each 
screen image of the display device is about Vy> second for 
dynamic pictures. 

Sound which is played back synchronously with the 
dynamic pictures can be delimited by Vx> second or so and 

55 stored with each page data which consists the dynamic 
picture. Therefore, the multi -media information playback 
device is equipped with a storage unit which stores sound 
data of at least Vso second so that the sound is successively 
played back along with the dynamic pictures. 

As for DVD. details can be found in the next reference: 
"Movies can be stored in a hard disc photo-electric mag- 
netic and phase change", featured in Jul. 18. 1994 issue of 
"Nikkei Electronics No. 613". pages 76-81. 

However, in the conventional art mentioned above, data 

65 about movies, sound, and their controlling, are stored fix- 
edly. Specifically, in the case of image data, data for the 
whole screen image including the background is stored as 
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one still picture data. Therefore, when pan of the screen 
image is changed by. e.g. moving a pan of the still picture 
to a different position on the screen, enlarging or reducing a 
pan of the still picture, more still picture data coping with 
the change has to be stored in the CD-ROM or the DVD. So . 5 
the maximum number of the still pictures which can be 
displayed is limited by the storage capacity of the CD-ROM 
and the DVD. Moreover, each time the still picture data is 
displayed, they have to be read from the CD-ROM and the 
DVD. so the number of still pictures which can be displayed ic 
per unit time is limited by the data transfer rate between the 
CD-ROM or the DVD and a memory. 

When the audio data, the still picture data, the dynamic 
picture data and the control information are stored in dif- 
ferent storage areas, the multi-media information playback ^ 
device has to access different storage areas of the CD-ROM 
or the DVD in order to read the control information and each 
data used for it every time it executes the control informa- 
tion. This results in bad read efficiency. Also, load of the 
multi-media information playback device for controlling the 20 ; 
internal memory is large since data are separately read into 
the memory provided to the multi-media information play- 
back device from the different storage areas of the CD-ROM 
and the DVD. 

25 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide a 
multi-media information playback device, in which readout 
of display data from a recording medium is not necessary 
each time a pan of a screen image is changed and displayed, 
the display data can be created easily, and the load on a CPU 
controlling a memory is small. 

The above object may be fulfilled by the raulu" -media 
information playback device which shows the following 35 
features. 

(1) A multi-media information playback device which 
plays back multi-media information recorded in a recording 
medium comprises a disc having page information succes- 
sively stored in a storage area, the page information includ- 40 
ing a partial image list having a pair of partial image 
information showing a partial image of a screen image and 
display state information showing display state of the partial 
image, and the page information also including control 
information which describes a command to be executed 43 
according to an input signal, the control informau'on includ- 
ing a display state change command instructing to change 
the display state of the partial image: a page information 
read unit for reading the page information from the disc; a 
display data creation unit for creating display Hafa of the 50 
screen image by placing the partial image shown by the 
partial image information on a screen; a display unit for 
displaying each display data created by the display data 
creation unit; an input unit for receiving an input signal from 
an operator, and a command execution unit for detecting the 55 
input signal inputted by the operator, judging whether the 
input signal corresponds to the command described by the 
control information in the page information read by the page 
information read unit, and instructing the display data cre- 
ation unit to create new display data by changing the display 
state of the corresponding partial image according to the 
display state change command when the input signal corre- 
sponds to the display state change command; wherein the 
display data creation unit creates new display data according 
to the command execution unit. o3 

According to the above construction, image information 
and display state of partial images constituting one screen 
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image arc shown by the page information successively 
recorded in the storage area on the disc. So in time of 
changing and displaying a part of a screen image, what is 
needed is only creating display data by changing the display 
5 state of the corresponding partial image according to the 
control information. The display data can be changed easily 
in this way. 

Also, a part of the screen image of display data can be 
changed only by the page information, so that the display 

JC data does not have to be read from the recording medium. So 
there is no need to record display data, whose part of the 
screen image is changed, in the recording medium before- 
hand. Therefore, the number of display data which can be 
displayed per unit time is not limited by the data transfer rate 
. l5 between the disc and the memory. 

Moreover, information needed for changing and display- 
ing only a part of the screen image using partial images is 
shown by the partial image list in one page information. So 
the load on a CPU which controls memory in the multi- 

20 media information playback device is small compared to the 
case in which necessary information is loaded to the 
memory of the multi-media information playback device. 
When a new image screen composed of different partial 
images in the multi-media information playback device is 

25 displayed, new page information which describes the partial 
images is read, and display data for showing that image 
screen is created from the page information. Therefore, new 
display data can be created by only one head seek which 
happens in time of reading new page information. Compared 

30 to the case in which partial image information is read from 
several places on the disc (which unit head seek happens 
several times), the readout of the partial image information 
can be carried out in short time. That means new image 
screen is displayed in shorter time than before. 

35 (2) The page information may include the partial image 
list, the control information, and the time sequence control 
information including a name of a second time sequence 
information as time sequence information to be played back 
according to the time sequence processing start command, 

40 pairs of IDs of sound information and image information to 
be played back at predetermined times, and the playback 
state information which describes at least one of dynamic 
picture and animation should be displayed in an area relating 
to the partial image shown by the partial image information; 

45 wherein the time sequence information may include the 
second time sequence information comprising sound infor- 
mation divided into equal predetermined sizes and image 
information showing an image of at least one of the dynamic 
picture and the animation, the sound information and the 

so image information being placed in order of playback corre- 
sponding to the elapsed time; wherein the time sequence 
processing execution unit may include a second time 
sequence processing execution unit for designating sound 
information and image information in the second time 

55 sequence information corresponding to the elapsed time at 
predetermined times when the second time sequence infor- 
mation is shown , by the time sequence processing control 
information, and instructing the display data creation unit to 
create new display data by placing the partial image read by 

60 the time sequence information read unit in an area shown by 
the playback state information: wherein the display data 
creation unit may create new display data according to the 
second time sequence processing execution unit 
According to above construction, the time sequence infor- 

65 mation including the second time sequence information, 
which describes sound information and image information 
in order of playback corresponding to the elapsed time, is 
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stored successively in the storage area on the disc. So head 
seek does not happen when the sound information and image 
information are successively read corresponding to the 
elapsed time. Therefore, the mulu-media information play- 
back device can read the sound information and the image 5 
information at high speed, so breaks does not happen in 
sound, animation and/or dynamic pictures, while the multi- 
media information is played back. 

(3 ) A multi-media information record device for recording 
multi-media information in a disc may comprise a partial I0 
image storage unit for storing the partial image information 
of at least one partial image in one partial image file, the 
partial image information being a part of the screen image: 
a partial image list input unit for receiving input of a name 
of at least one partial image file in the partial image storage l5 
unit and display state information showing display state of 
the partial image shown by the partial image information In 
the partial image file: a page information storage unit for 
having a plurality of separate small storage areas, each of 
which is used for storing at least a piece of page information: 2c 
a partial image list write unit for writing a pair of the partial 
imace information and the display state information in a first 
small storage area in the page information storage unit along 
with the IDs of the partial image information and the display 
state information, after reading partial image information 25 
from the partial image file whose name is inputted, the 
display state information being inputted according to the 
partial image informaUon: a control information input unit 
for receiving input of control information which includes a 
name of input signal and display state change command 3C 
instructing to change display state of the partial image 
corresponding to the input signal: a control information 
write unit for writing the inputted control information in a 
second small storage area in the page information storage 
unit; and a page information record unit for successively 35 
recording information, which is written as a piece of page 
information in each small storage area in the page informa- 
tion storage unit, in a storage area on the disc. 

According to the above construction, the page informa-. 
tion can be stored successively in the storage area on the 40 
disc. Effects mentioned above can be obtained from the 
multi-media information playback device which plays back 
multi-media information from the disc. 

(4) A multi-media information playback device for read- 
ing multi-media information from a data server connected 45 
via a network and playing back the multi-media information 
may comprise a page information read unit for reading a 
piece of page information stored as one file from a data 
server connected via a network, the page information includ- j 
ing a partial image list having at least one pair of the partial 50 
image information showing a partial image of a screen 
image and display state information describing display state 
of the partial image, and the page information also including 
control information which describes a command to be 
executed according to an input signal, the control informa- 35 
tion including a display state change command instructing to 
change the display state of the partial image: a display data 
creation unit for creating display data of the screen image by 
placing the partial image shown by the partial image infor- 
mation on a screen: a display unit for displaying each display «c 
data created by the display data creation unit: an input unit 
for receiving an input signal from an operator, and a 
command execution unit for detecting the input signal 
inputted by me operator, judging whether the input signal 
corresponds to the command described by the control infor- 65 
mation in the page information read by the page information 
read unit, and instructing the display data creation unit to 
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create new display data by changing the display state of the 
corresponding partial image according to the display state 
change command when the input signal corresponds to the 
display state change command; wherein the display data 
5 creation unit may create new display data according to the 
command execution unit. 

According to the above construction, effects mentioned 
above can be obtained when mu It i- media information is read 
from the data server connected via a network and played 
io back. 

(5) A multi-media information record device for recording 
multi-media information in a data server connected via a 
network may comprise a partial image storage unit for 
storing the partial image information of at least one partial 
15 image in one partial image file, the partial image information 
being a part of the screen image: a partial image list input 
unit for receiving input of a name of at least one partial 
image file in the partial image storage unit and display state 
information showing display state of the partial image 

20 shown by the partial image information in the partial image 
file; a page information storage unit for having a plurality of 
separate small storage areas, each of which is used for 
storing at least a piece of page information: a partial image 
list write unit for writing a pair of the partial image infor- 

25 mation and the display state information in a first small 
storage area in the page information storage unit along with 
the IDs of the partial image information and the display state 
information, after reading partial image information from 
the partial image file whose name is inputted, the display 

30 state information being inputted according to the partial 
image information: a control information input unit for 
receiving input of control information which includes a 
name of input signal and display state change command 
. instructing to change display state of the partial image 

35 corresponding to the input signal: a control information 
write unit for writing the inputted control information in a 
second small storage area in the page information storage 
unit; and a page information record unit for recording 
contents of a piece of page information in one file in the data 

40 server after transmitting contents written as a piece of page 
information in each storage area in the page information 
storage unit to a data server along with its write request. 

According to the above construction, each page informa- 
tion can be stored in individual file in the data server, and 

45 above effects can be obtained in the multi-media information 
playback device which reads and plays back* multi-media 
information from the data server. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other objects, advantages and features of the 
invention will become apparent from the following descrip- 
tion thereof taken in conjunction with the accompanying 
drawings which illustrate a specific embodiment of the 
invention. In the drawings: 

FIG. 1 is an example of data configuration of multi-media 
information, which is showing synchronous playback of 
images and sound in the conventional multi-media informa- 
tion playback device. 

FIG. 2 is a block diagram showing functional construction 
of a multi-media information playback device 200 of the 
first embodiment of the present invention. 

FIG. 3 is a block diagram showing hardware construction 
of the multi-media information playback device 200 shown 
65 in FIG. 2. 

FIG. 4 shows how multi-media information is recorded in 
the CD-ROM 201. the multi-media information being 
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flayed back by the multi-media information playbackdevice 
200 of the present invention. 

FIG. 5 shows data configuration of a page stream played 
back by the multi-media information playback device 200 of 
the present embodiment. 5 

FIG. 6 shows data configuration of time line stream 
played back by the multi-media information playbackdevice 
200 of the present embodiment. 

FIG. 7 is a flow chart showing procedure of playback of 
the multi-media information in the multi-media information A ° 
playback device 200 of the present embodiment. 

FIG. 8 shows contents of the page stream 1. which is a 
concrete example of the page stream shown in FIG. 5. 

FIG. 9(a) shows contents of the initial image illustrated by 15 
a display data creation unit 213 based on the partial image 
list of the page stream 1 shown in FIG. 8. 

FIG. 9(b) shows contents when an input signal "A" is 
inputted while the page stream 1 is executed 

FIG. 9(c) shows contents right after an input signal "B" in 20 
inputted while the image shown in FIG. 9(b) is displayed. 

FIG. 9(d) shows contents at "00 minute 01.000 second", 
which is an elapsed time after the start of the execution of 
the time line information. 

FIG. 9(e) shows contents at "00 minute 02.000 seconds" 25 
which is an elapsed time after the start of the execution of 
the time line information. 

FIG. 9(f) shows contents of the initial image when the 
page stream 2 is loaded to the page storage unit 210. 30 

FIG. 10 shows contents of a time line stream 1. which is 
a concrete example of the time stream shown in FIG. 6. 

FIG. 11 shows contents of a page stream 2. which follows 
the page stream 1 shown in FIG. 8. 

FIG. 12 is a block diagram showing construction of the 33 
multi-media information record device 1200 of the second 
embodiment of the present invention. 
^ FIG. 13 is a block diagram showing hardware construc- 
tion of the multimedia information 1200 shown in FIG. 12. 40 

FIG, 14 is a flow chart showing procedure of the record- 
ing of the multi-media information in the multi-media 
information record device 1200 of the present embodiment. 

FIG. 15 is a flow chart showing procedure of the record- 
ing processing of the multi-media information in the multi- 45 
media information record device 1200 of the present 
embodiment. 

FIG. 16 is a block diagram showing construction of the 
multi-media information playback device 1600 of the third 
embodiment of the present invention. 50 

FIG. 17 is a block diagram showing hardware construc- 
tion of the multimedia information 1600 shown in FIG. 16. 

FIG. 18 is a block diagram showing construction of the 
multi-media information record device 1800 of the fourth 
embodiment of the present invention. 55 

FIG. 19 is a block diagram showing hardware construc- 
tion of the multimedia information 1800 shown in FIG. 18. 

DESCRIPTION OF THE PREFERRED 

EMBODIMENTS 60 

(Embodiment I) 

FIG. 2 is a block diagram showing functional construction 
of a multi-media information playback device 200 .of 
Embodiment 1 of the present invention. FIG. 3 is a block 6S 
diagram showing hardware construction of the multi-media 
information playback device 200 shown in FIG. 2. 



8 

As shown in FIG. 2. the multi -media information play- 
back device 200 comprises a CD-ROM 201. a CD-ROM 
interface unit 202. an input unit 203. a control information 
execution unit 204. a clock unit 205. a page load unit 206. 

5 a time line control unit 207. a storage unit 208. a display unit 
212. a display data creation unit 213. an audio D/A conver- 
sion unit 214, and an audio output unit 215. The storage unit 
208 further comprises a display data storage unit 209. a page 
storage unit 210. and an audio data storage unit 211. 

i0 As shown in FIG. 3. the multi-media information play- 
back device 200 comprises a CPU 301. a RAM 302, a 
system clock 303. a timer 304. a key board 305. a joy pad 
306. an input device interface 307. a disc interface 308. a 
CD-ROM 309. a MPEG decoder 311. a sound D/A convenor 

15 312. a speaker 313. a graphic controller 314. a VRAM 315. 
an image D/A convenor 316. and a CRT 317. as components 
of hardware. The CPU 301 comprises functions to control 
operation of each component of the multi-media information 
playback device 200. in addition to the functions which are 

20 explained as follow. 

The CD-ROM 201 can be attached/detached to the multi- 
media information playback device 200. The following 
multi-media information is written by High Sierra Format of 
2s ISO 9690 and the like. 

FIG. 4 shows how multi-media information, which is 
played back by the multi-media information playback device 

200 of the present embodiment, is stored in the CD-ROM 
201. Storage areas 401. 402. and 403 of the CD-ROM 201 

3Q continues successively on the track, and data stored in each 
of the storage areas is read one after another as the CD-ROM 

201 turns. A page stream 1. which is mainly composed of 
partial image data and mentioned later, is recorded in the 
storage area 401. A page stream 2. which is mainly com- 

35 posed of partial image data and shows the page following the 
page stream 1. is recorded in the storage area 402. A time 
line stream 1. which is mainly composed of data about 
dynamic pictures and animations relating to the page stream 
1. is recorded in the storage area 403. and mentioned later. 

^ Data is read from the CD-ROM 201 while the page stream 
1. the page scream 2, and the time line stream 1 are used as 
a logical unit, respectively. But they don't necessarily cor- 
respond to the physical unit of the CD-ROM 201. like a 
sector. 

45 A page is an image to be displayed on the screen, which 
is displayed every Yys second or so. Each page stream is 
multi-media information which describes contents of a plu- 
rality of pages. Each of the pages is composed of one or a 
plurality of predetermined partial images. If the original 

50 partial image is the same as the partial image which is 
mentioned later by the page stream, a partial image can be 
enlarged/reduced or can be moved parallel/rotationally and 
one or a plurality of partial images can be deleted by the 
same page stream. 

55 FIG. 5 shows data configuration of a page stream which 
is played back by the multi-media information playback 
device 200 of the present embodiment. A page stream 
consists of a partial image list, control information, and rime 
line information. Each page stream is given an identification 

60 number. 

In the partial image list, contents of each partial image 
consisting of one screen image is written: a partial image ID 
which designates a partial image; X-Y coordinates which 
show the initial position of the partial image when it is 
65 displayed in the page; X-Y enlargement rates which show 
the scale of enlargement or reduction when the partial image 
is first displayed in the page; a rotational angle between a 
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panial image shown by the partial image data and the partial 
image displayed in the page; and partial image data which is 
expressed by the values corresponding to the pixels of the 
partial image. The partial image ID is expressed by the 
number which shows order of placing the partial images in 5 
the display data, when display data of one screen image is 
created by the partial images shown by the page stream. 
Therefore, when the display areas of the partial image 
partially overlap with each other, the partial image data 
corresponding to the pixels of the overlapping areas is l0 
updated by partial image data which has a greater partial 
image ID. So partial images are displayed according to the 
partial image ID. which means that the partial image having 
a greater partial image ID covers the proceeding partial 
images. The partial image data shown by the page stream 15 
can be expressed by compressed and decoded bit map data 
and. if the partial image is geometrical graphic form, it can 
be expressed by graphic data. Also, the partial image docs 
not necessarily have to be a graphic form. It can be a string 
showing an input request displayed on each menu screen. In ^ 
this case, the string can be shown by a character code. 

The control information describes commands to be 
executed by the multi-media information playback device 
200 for the input signal from the input unit 203 while the 
page shown by the page stream is displayed. The commands M 
are described for each input signal. The commands are: 
change the X-Y coordinates of the partial image: change the 
X-Y enlargement rates of the partial image; change the 
rotatioaal angle of the partial image; load other page 
streams; start the execution of the time line information; 3c 
overlap a plurality of partial images; display the color 
images in black and white; emphasize a specific color; 
change frequency of sound; and the like. 

T>: dme line information describes animation display and 
dynamic picture display; the animation display moves pre- 35 
determined image pictures shown by the partial image list as 
time goes by: the dynamic picture display displays the 
dynamic pictures shown by the time line stream mentioned 
later. Specifically, as for the animation display, the following 
items arc shown: a partial image ID of the partial image to 40 
be displayed in animation (animation object); X-Y coordi- 
nates which shows a predetermined display position at every 
given time after the start of the animation display of the 
partial image; and an animation display end time of the 
partial image. As for the dynamic picture display, the fol- 45 
lowing items are shown: the identification number of the 
time line scream which shows images to be displayed in 
dynamic picture; and a start time and an end time of the 
dynamic picture display. Also, when the dynamic picture 
image is displayed inside the predetermined partial image 50 
shown by the partial image list, the partial image ID of the 
partial image which is displayed in dynamic picture is 
shown so that the dynamic picture is displayed in place of 
the predetermined partial image. The dynamic picture image 
shown by the time line stream mentioned later does not 55 
necessarily have to be displayed inside the redetermined 
partial image shown by the partial image list 

FIG. 6 shows data configuration of the time line stream 
which is played back by the multi-media information play- 
back device 200 of the present embodiment. Identification 60 
numbers are given to each time line stream. As described 
already, dynamic pictures and animations are displayed by 
playback of the successive still pictures at every display 
time, which is every V60 second or so. In the time line stream 
shown in FIG. <6. still picture data expressed by the com- 65 
pressed and encoded bit map data is stored according to the 
order of the display, along with the audio data corresponding 
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to ihe still picture. Also the audio data is compressed, 
encoded, divided into predetermined equal sizes correspond- 
ing to the display rime of the still picture data, and played 
back successively while the still picture data corresponding 

5 to the audio data is displayed. Each display time can be 
expressed by the elapsed time after the start of the clock unit 
205. When the clock 205 can measure the elapsed time from 
the start of the display for a plurality of time streams, the 
display time can be expressed by the e;apsed lime from the 

10 start of the execution of each lime line stream. 

Also the audio data and the still picture data do not 
necessarily have to be shown by the time line stream. For 
example, when the audio data shown by the time line stream 
is played back while the display picture is displayed by the 

15 partial image shown by the page stream, only audio data can 
be shown in the time line stream. Animations as w f ell as 
dynamic pictures can be shown by this time line stream. So 
in order to distinguish these animations from the animations 
displayed by the parti aJ image shown by the page stream, a 

20 term "dynamic picture display" is used for the display of 
animations and dynamic pictures using the still picture data 
shown in the time line stream. 

The CD-ROM interface unit 202 is implemented by the 
disc interface 308. and reads multi-media information into 

25 the multi -media information playback device 200 according 
to the instruction of the page load unit 206. the multi-media 
information being described by the page stream and the time 
line stream recorded in the CD-ROM 201. 

3c The input unit 203 comprises the joy pad 306. the key 
board 305. and operation unit like a mouse, which is not 
illustrated, and receives input of instruction from a user via 
the input device interface 307. The input unit 203 outputs the 
input signal corresponding to the received input. For 
example, this unit 203 outputs the input signals A. B. and C, 
which correspond to the buttons A. B. and C in the joy pad 
306. to the control information execution unit 204. 

The control information execution unit 204 is imple- 
mented by the CPU 301. This unit 204 interprets and 

^ interactively executes control information shown by the 
page stream loaded to the page storage unit 210. Specifically, 
the control information execution unit 204 instructs the page 
load unit 206. the time line control unit 207. and the display 
data creation unit 213 to execute processing based on the 

A . instruction of the control information. 

The clock unit 205 is implemented by the system clock 
303 and the timer 304. Being activated by the time line 
control unit 207. the timer 304 measures the elapsed time 
from the execution start of the time line information with 

30 reference to the system clock 303. The clock unit 205 can be 
equipped with a plurality of the rimers 304 so that the 
elapsed time from the start of the execution of each time line 
stream can be measured along with the elapsed time from the 
start of the execution of the time Line information. 

55 The page load unit 206 is implemented by the CPU 301. 
This unit 206 instructs the CD-ROM interface unit 202 to 
read out a predetermined page stream and loads the page 
stream to the page storage unit 210. when the multi-media 
information playback device 200 is activated. This unit also 

6o instructs the CD-ROM interface unit 202 to read the desig- 
nated page stream and the time line stream according co the 
instruction of the time line control unit 207 or the control 
information execution unit 204. and loads the page stream or 
the time line stream to the page storage unit 210. 

65 The time line control unit 207 is implemented by the CPU 
301. and starts the execution of the time line information 
described by the loaded page stream according to the control 
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information execution unit 204. This unit 207 activates the 
clock unit 205 when the execution of the time line infor- 
mation is started. This unit 207 initializes the timer 304 of 
the clock unit 205 when the execution of the time line 
information is completed. In case each time line stream is < 
equipped with the timer 304 which measures the elapsed 
time from the start of the execution, corresponding timer 304 
is initialized when execution of each time line stream is 
completed. 

The time line control unit 207 instructs the display data ic 
creation unit 213 to place designated partial images, carry- 
ing out the designated processing like enlarging/reducing 
and rotational migrating, at the designated position of the 
X-Y coordinates, according to the elapsed time from the start 
of the execution of the time line information measured by 15 
the clock unit 205 when the time line information which 
instructs animation display is executed. 

The time line control unit 207 instructs the page load unit 
206 to load the time line stream designated in the time line 
information to the page storage unit 210. when the time line 20 
information which instructs the dynamic picture display is 
executed. This unit 207 designates the still picture data 
corresponding to the elapsed time from the start of the , 
execution of the time line information measured by the clock 
unit 205 and posts the designated still picture data to the 25 
display data creation unit 213. Then this unit 207 instructs ' 
the display data creation unit 213 to place the still picture 
data at the predetermined position, for example, inside the 
partial image designated in the time line information. At the 
same time, this unit 207 reads the audio data corresponding 30 
to the still picture data from the time line stream, and 
transfers the audio data to the audio data storage unit 211. 

J] 1 ? *I S £ lay St0rage Uflit 209 is implemented by the 
VRAM 315. and stores the display data of one screen page 
which is created by the display data creation unit 213. 35 

The page storage unit 210 is implemented by the RAM 
302. and stores the page stream and the time line stream read 
from the CD-ROM 201. 

t>??™ di0 ? ata "ora^unit 211 is implemented by the M 
RAM 302. and stores audio data which is divided into about 
l /» second sizes. 

The display unit 212 is implemented by the image D/A 
convener device 316 and the CRT 317. This unit 212 reads 
the display data stored in the display data storage unit 209 45 * 
eyery Vx> second or so and displays it in the CRT. The image 
D/A converter 316 reads the display data from the VRAM 
315. which is the display data storage unit 209. every '/» 
second or so. converts the display data to analog signals, and ] 
supplies them to the CRT 317. The CRT 317 displays the 
analog signals which are converted from the display data In 
the present embodiment, the CRT 317 is used for the display 
unit 212. but a liquid crystal display LCD or a television 
which is connected to the multi-media information playback 
device 200 can substitute for the CRT 317. The contents of 55 
the display unit 212 can be printed on papers. 

The display data creation unit 213 is implemented by the 
CPU 301. the MPEG decoder 311. and the graphic controller 
314. This unit 213 creates display data of a whole screen of 
an initial image, using the partial images shown by the 60 
loaded page stream, when a new page steam is loaded to the 
page storage unit 210. According to the instruction of the 
control information execution unit 204. this unit 213 carries 
out the designated processing (e.g. moving X-Y coordinates, 
enlarging/reducing, rotational moving, etc.) for the partial 65 
images shown by the loaded page stream, and creates 
display data of a whole screen which shows a still image 
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using the partial images. According to the instruction of the 
time line control unit 207. the unit 213 carries out the 
designated processing mentioned above for the partial 
images shown by the loaded page stream, creates display 

5 data for animation display, and update the display data for 
dynamic picture display, using the still picture data desig- 
nated by the time line control unit 207 from the still picture 
data shown by the designated time line stream. At this point, 
the MPEG decoder 311 expands and decodes the image data. 

ic which was compressed and encoded according to the 
MPEG, such as still picture data shown by the time line 
stream and partial image data shown by the page stream. 

The audio D/A converter unit 214 is implemented by the 
MPEG decoder 311 successively and the sound D/A con- 

15 verter 312. The MPEG decoder 311 reads the audio data 
stored in the audio data storage unit 211. and expands and 
decodes the audio data, which is stored compressed and 
decoded according to the MPEG. The sound D/A converter 

312 converts the decoded audio data into analog signals, and 
20 outputs them to the speaker 313. As well as the image data. 

the audio data does not necessarily have to be compressed 
and decoded according to the MPEG. The audio data can be 
compressed and decoded according to other formats, or does 
not have to be compressed and decoded at all. Therefore, the 
25 MPEG decoder 311 can be substituted for other decoders. 
When audio data which is not compressed and decoded is 
treated, there's no problem without decoders. 
The audio output unit 215 is implemented by the speaker 

313 and converts analog signals from the audio D/A con- 
30 verter unit 214 to sound and outputs the sound. 

Explanation about the procedure of the playback of multi- 
media information in the multi-media information playback 
device 200 follows next: it refers to a flow chart in FIG. 7; 

3j concrete examples of the page stream 1, the page stream 2 
and the time line stream illustrated from FIGS. 8-11; and 
concrete example of the contents displayed based on them. 

FIG. 7 is a flow chart showing the procedure of the 
playback of multi-media information in the multi-media 

^ information playback device 200 of the present embodi- 
ment 

The page load unit 206 instructs the CD-ROM interface 
unit 202 to read a predetermined page stream, like page 
stream 1 mentioned later, from the CD-ROM 201. and loads 
45 the read page stream to the page stream storage unit 210. 
(Step 701). 

The display data creation unit 213 creates display data of 
the initial image from each partial image shown by the 
partial image list of the loaded page stream (Step 702). The 

so display data is transferred to the display data storage unit 
209, stored there, read and displayed by the display unit 212 
every '/» second or so. 

FIG. 8 shows contents shown by the page stream 1. which 
is a concrete example of the page stream shown in FIG. 5. 

55 In the partial image list, a circle having a pattern of oblique 
lines inside, a triangle having a pattern of vertical dashed 
lines inside, and a rectangle having no partem inside are 
shown by the bit map data or graphic data as partial image 
data, according to the order of the partial image IDs. Also the 

60 X-Y coordinates which shows the display position of the 
partial image, the X-Y enlargement rales of the partial 
image, and a rotational angle are shown. In case of a circle 
whose partial image ID is 1. bit map data shown by the 
partial image data is placed based on the X-Y coordinates 

65 (100. 100) by the display data creation unit 213. and display 
data is created with the enlargement rates being (1.1). and 
the rotational angle being 0. The display data of the triangle 




and the rectangle whose partial image IDs are 2 and 3. 
respectively, is created in the same way mentioned above. So 
in the areas where each partial image overlaps, the partial 
image data having a small partial image ID is updated by the 
partial image data having a greater partial image ID. 5 

FIG. !>*,a) shows the contents of the initial image, which 
is illustrated by the display data creation unit 213. based on 
the partial image list of the page stream 1 illustrated in FIG. . 
8. As mentioned already, in the illustration processing of the 
display data creation unit 213. the partial image data having io 
a smaller partial image ID is updated by the partial image 
having a greater partial image ID in areas where each partial 
image overlaps. So. as illustrated in the initial image of the 
FIG. 9(a). partial images having a greater partial image ID 
are displayed on top of the layers of other partial images, 

The control information execution unit 204 detects an 
input signal from the input unit 203. when the processing of 
Step 702 is completed (Step 703;. and starts Step 708 when 
the input signal is not detected. 

When the input signal is detected, the control information 2C 
execution unit 204 identifies the input signal (Step 704). and 
judges whether the input signal is shown by the control 
information of the page stream (Step 705). When the input 
signal is not shown, the control information execution unit 
204 starts Step 708. 25 

The control information execution unit 204 executes the 
command which is described according to the input signal, 
when the identified input signal is judged to be shown by the . 
control information of the page stream (Step 706). The 3Q 
display data creation unit 213 creates display data based on 
the execution result of the control information execution unit 
204 (Step 707). 

The following explanation is about the processing from 
Steps 705-707 in the multi-media information playback 35 
device 200. referring to concrete examples. 

The control information execution unit 204 identifies an 
input signal, and reads the control information from the page 
stream 1 in FIG. 8. which is loaded to the page storage unit 
210. The control information is shown as A. B. and C. so that ^ 
the control information execution unit 204 can judge the 
type of the input signal. 

The control information execution unit 204 executes a 
command "The X-Y coordinates of the partial image 1 
should be ( 170. 100)". That is because when the identified 45 
input signal is A. it is shown by the control information. 
Specifically, the control information execution unit 204 
passes parameters "partial image ID=1". and "X-Y coordi- 
nates (170. lOOy to the display data creation unit 213. The 
control informadon execution unit 204 instructs the display 5c 
data creation unit 213 to create display data based on the 
parameters. The display data creation unit 213 creates the 
display data according to the instruction. FIG. 9(b) shows 
contents when input signal A is inputted while the page 
stream 1 is executed. The partial image 1 is moved from 55 
(100. 100) to (170. 100). 

When the identified input signal is B. the control infor- 
mation execution unit 204 executes a command 'The clock 
of the time line should be started.", which corresponds to the 
input signal B. Specifically, the control information execu- « 
tioo unit 204 instructs the time line control unit 207 to start 
the execution of the time line information shown by the page 
stream 1. Then the time line control unit 207 activates the 
clock unit 205. The time line control unit 207 also reads the 
time line information by the page stream 1. and when mere's 65 
is a designated time line stream in the time line information 
which shows the dynamic picture display, the time line 
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control unit 207 instructs the page load unit 206 to load the 
time line stream. The time line control unit 207 further 
creates parameters mentioned later based on the time line 
information, and passes it to the display data creation unit 
5 213. and instructs the display data creation unit 213 to create 
display data based on the parameters. 

As is shown in FIG. 8. time line information about the 
animation display and the dynamic picture display is shown 
by the page stream 1. 
10 The following items are shown concerning the animation 
display: a partial image ID "3" of the partial image to be 
displayed in animation (animation subject): 00 minute 
00.000 second as an animation display start time and 00 
minute 03.000 seconds as an animation display end time. 
15 both of which are expressed by the elapsed times from the 
start of the execution of the time line information: X-Y 
coordinates (150. 150) at 00 minute 00.000 seconds, X-Y 
coordinates (170. 170) at 00 minute 01.000 seconds. X-Y 
coordinates ( 190. 150) at 00 minute 02.000 seconds, all of 
2C which show the display positions of the partial image 3 at 
predetermined elapsed times. 

The following items are shown concerning the dynamic 
picture display: an identification number "1" of the time line 
stream to be displayed in dynamic pictures; 00 minute 
25 00.000 second as a dynamic picture display start tirae and 00 
minute 03.000 seconds as dynamic picture display end time, 
both of which are expressed by the elapsed times from the 
start of the execution of the time line information. 

The time line control unit 207 instructs the page load unit 
206 to load the time line stream 1 to the page storage unit 
210. FIG. 10 shows contents of the time line stream 1. which 
is a concrete example of the lime line stream shown in FIG. 
6. Still picture data which shows the rising sun and audio 
data to be played back while the still picture data is dis- 
3 played are shown by the time line stream, both of which 
correspond to each elapsed time between 00 minute 00.000 
second and 00 minute 03.000 seconds. 

When the identified input signal is C. the control iofor- 
mation execution unit 204 executes a command "Read the 
page stream 2". which corresponds to the input signal C. 
Specifically, the identified input signal instructs the page 
load unit 206 to load the page stream 2 to the page storage 
unit 210. Then the page load unit 206 instructs the CD-ROM 
45 interface unit 202 to read the page stream 2, and loads the 
read page stream 2 to the page storage unit 210. The display 
data creation unit 213 creates display data based on the 
partial image list of the loaded page stream 2. 

FIG. 11 shows contents of the page stream 2. which 
5C follows after the page stream 1 shown in FIG. 8. In FIG. 11. 
a rectangle having a pattern of oblique lines inside is shown 
by the partial image data as bit map data, along with a partial 
image data ID "1". X-Y coordinates (100. 100). X-Y 
enlargement rates (1.1). a rotational angle **0*\ Also, a 
55 command for the input signal A 'The X-Y" coordinates of the 
partial image I should be (200. 200)" is described by the 
control information of the page stream 2. Therefore, when A 
is inputted while the page stream 2 is executed based on the 
partial image list mentioned above, the partial image 1 is 
6c placed at (200. 200) of the X-Y coordinates. A time line 
information is not shown by the page stream 2. 

FIG. 9. (/) shows contents of the initial image when the 
page stream 2 is loaded to the page storage unit 210. The 
rectangle having oblique lines inside, which is shown by the 
65 partial image list of the page stream 2. is shown by X-Y 
enlargement rates (1.1). a rotational angle ~0*\ and X-Y 
coordinates (100. 100). 




15 _ 

The time line control unit 207 checks the clock unit 205 
to see whether it is operating, when Step 707 finishes (Step 
708). When the clock unit 205 is operating, the time line 
control unit 207 reads the elapsed time from the start of the 
execution of the time Line information, which is measured by 5 
the clock unit 205 (Step 709). 

The time line control unit 207 judges whether there is a 
partial image designated to be displayed in animation, by the 
elapsed time read from the clock unit 205 (Step 710). When 
there is such a partial image, the time line control unit 207 io 
calculates X-Y coordinates, which shows the display posi- 
tion of the partial image, from the already designated X-Y 
coordinates which correspond to a predetermined elapsed 
time. Then the time line control unit 207 passes the calcu- 
lated X-Y coordinates. X-Y enlargement rates, a rotational 15 
angle and a partial image ID of the partial image to the 
display data creation unit 213 as parameters. Then the time 
line control unit 207 instructs the display data creation unit 
213 to create display data based on the parameters. The 
display data creation unit 213 creates display data based on 20 
the parameters, and updates the display data in the display ' 
data storage unit 209 (Step 711). When there's no designated 
partial image in Step 710. the time line control unit 207 starts 
Step 712. 

For example, when the elapsed time in Step 709 is 00 25 
minute 00.000 second, the animation display start time 
shown by the time line information is 00 minute 00.000 
second. So the time line control unit 207 creates a partial 
" r ^ X " Y coordiQat « (150.150)) at 00 minute 
00.000 second as parameters for animation display, and 30 
passes them to the display data creation unit 213. Then the 
display data creation unit 213 moves the partial image 3 to 
the position designated by the parameter "the X-Y coordi- 
aates ( 150.150) at 00 minute 00.000 second", creates display 
data, and updates the display data in the display data storace 33 
unit 209. ^ 

Also, when an elapsed time is 00 minute 01.000 second in 
Step 709. the partial image 3. which is designated to be 
displayed in animation, exists. So the time line control unit 
207 creates an animation display partial image 3D "3" and 40 
X-Y coordinates (170.170) at 00 minute 01.000 second as 
parameters for animation display. Then this unit 207 passes 
them to the display data creation unit 213. The display data 
creation unit 213 moves the partial image 3 to (170, 170) 
based on the parameters and creates display data, and 45 
updates the display data in the display data storage unit 209. 

When the elapsed time is 00 minute 00.500 second/ the 

^ eO ^ 0i Uail 207 calcula '« "X-Y coordinates 
(160.160) at 00 minute 00.500 second" which is interpolat- 

n 8 j£ tWecn thC X " Y coordinates (150.150) at 00 minute 
At aaa SCCOnd 3Qd X * Y coordinat « (170. 170) at 00 minute 
01.000 second, and passes it to the display data creation unit 
213. The display data creation unit 213 moves the partial 
image to the (160. 160); creates display data and updates the «< 
display data in the display data storage unit 209. 

Moreover, the time line control unit 207 judges whether 
there is a time line stream designating dynamic picture 
display . by the elapsed time read from the clock 205 (Step 
712). When the designated time line stream exits, the time 60 
hue control unit 207 designates the still picture data corre- 
sponding to the elapsed time read from the clock unit 205 
and posts the data to the display data creation unit 213. Also 
the time line control unit 207 reads audio data corresponding 
to the still picture data from the time line stream and 65 
transfers that data to the audio data storage unit 211. The 
display data creation unit 213 reads the designated still 
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picture data from the designated time line stream and creates 
display data. After the time line control unit 207 updates the 
display data in the display data storage unit 209 (Step 713). 
this unit starts Step 703. If the designated time line stream 

5 does not exist in Step 712. the time line control unit 207 
starts Step 703. 

For example, when the elapsed time in Step 709 is 00 
. minute 00.000 second- the dynamic picture display start time 
. shown by the time line information is 00 minute 00.000 

10 second. So the time line control unit 207 creates identifica- 
tion number w r of the time line stream. 00 minute 00.000 
second of the still picture data, and the partial image ID 
as parameter for the dynamic picture display corresponding 
to the elapsed time 00 minute 00.000 second from the start 

15 of the execution of the time line information, and passes 
them to the display data creation unit 213. At the same time, 
the time line control unit 207 reads audio data Al from the 
time line stream 1. and transfers it to the audio data storage 
unit 211. The time line control unit 207 instructs the display 

2C data creation unit 213 to create display data based on the 
parameter. Then the display data creation unit 213 places the 
still picture data designated by the parameter "still picture 
data=00 minute 00.000 second" inside partial image 3. 
creates display data, and updates the display data in the 

23 display data storage unit 209. 

FIG. 9(c) shows contents right after the input signal B is 
inserted while FIG. 9(b) is displayed. FIG. 9(c) shows a 
screen image created based on each parameter whose 
elapsed time measured by the clock unit 205 is 00 minute 

3C 00.000 second. The partial image 3 to be displayed in 
animation is displayed by "X-Y coordinates (150. 150) at 00 
minute 00.000 second", while the partial images 1 and 2 are 
displayed on the same position as FIG. 9(b). since their X-Y 
coordinates are not designated by parameters for animation 

35 display. Also, still picture data of "still picture data=00 
minute 00.000 second" in the time line stream 1 is displayed 
Inside the partial image 3. At the same time, the audio data 
Al is played back. The difference between the contents of 
the initial image shown in FIG. 9(a) and that of FIG. 9(c) is 

40 the position of the circle having a pattern of oblique lines. In 
the former case, the circle is shown by the X-Y coordinates 
(100. 100); in the latter case, it is displayed by (170. 100). 

Moreover, when the elapsed time in Step 709 is 00 minute 
01.000 second, the time line stream 1 designating dynamic 

45 picture display, exists. So the time line control unit 207 
creates parameters "identification number of the time line 
stream= 1", "still picture data=00 minute 01.000 second-, 
and "partial image ID=3" corresponding to the elapsed time 
"00 minute 01.000 second*' for dynamic picture display, and 

30 passes them to the display data creation unit 213. At the 
same time, the time line control unit 207 reads the audio data 
Ai corresponding to the elapsed time 00 minute 01.000 
second, from the time line stream 1. and transfers the data to 
the audio data storage unit 211. The display data creation 

55 unit 213 places the still picture data of the "00 minute 01.000 
second", which is read from the time line stream 1. inside the 
partial image 3 according to the parameter, and creates 
display data. 

FIG. 9(d) shows contents at 00 minute 01.000 second. 

60 which is the elapsed time from the start of the execution of 
the time line information. As shown in FIG. 9(d), the partial 
images 1 and 2 are displayed at the same positions as FIG. 
9(b). but the partial image 3 is moved to the position of the 
X=Y coordinates (170. 170). and the still picture data of TO 

65 minute 01.000 second" is displayed inside the partial image 
3. While this image is displayed, the audio data Ai is played 
back. 
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Moreover. FIG- 9 shows contents at the elapsed time. 00 
minute 02.000 seconds. The partial images 1 and 2 are 
displayed at the same positions as they are displayed in FIG. 
9{e). by the same processing mentioned above. The partial 
image 3 is move to ( 190. 150), and a still picture data of "00 5 
minute 02.000 seconds" are displayed inside it. While this 
still picture is displayed, the audio data Aj is played back. 
X-Y coordinates for animation display for the partial image 
3 after "00 minute 02.000 seconds" is not shown by the 
control information in the page stream 1. While animatioa 10 
display continues until "00 minute 03.000 seconds", the 
partial image 3 is displayed at the X-Y coordinates (190. 
150). However, still picture data until M 00 minute 03.000 
seconds'* are successively displayed inside the partial image < 
3. and the audio data is also played back at the same time. 15 

As explained above, display of partial images can be 
changed easily according to the present embodiment, com- 
pared to the case in which contents of one screen is stored, 
as one image data in the recording medium. Also, even 
though the contents are limited to the combinations of the 
partial images, maximum number of the still pictures to be 
displayed can be increased regardless of the storage capacity ■ 
of the recording medium. In case of changing the display of 
the partial image shown by in the loaded page stream, as the 
image data does not have to be read from the recording 2 $ 
medium, the multi-media information playback device 200 
can display many still pictures per unit time, regardless of 
. the transfer rate. The contents of each page stream and each 
time line stream is successively stored on the same crack of 
the CD-ROM. so head seek does not happen when the 30 
information is successively read. Speed of data readout from 
the same page stream and the same time line stream is fast 
and breaks in the sound and dynamic pictures do not happen, 
so that the multi-media information is played back accu- 
rately. As the partial image list, the control information 35 
relating to the partial image, and the time line information 
are described by one page stream, the image data, control 
information, and dynamic information are loaded to the 
RAM 302 and the load of the CPU is decreased, unlike the 
case in which the information are recorded in the different 4c 
storage areas in the recording medium. 

In the present embodiment, the still picture data and the 
audio data shown by the time line stream are compressed 
and encoded according the MPEG, and they are expanded 
and decoded by the MPEG decoder 311. But the still picture 45 
data, the audio data and the partial image data are not 
necessarily have to be compressed and encoded according to 
the MPEG. They can be compressed and encoded by other 
formats like VFW (Video for Windows) or QT (Quick 
Time). Therefore the MPEG decoder 311 can be substituted 5c 
for decoders following other formats, or a decoder is not 
necessary at all in case only the data which is not com- 
pressed and encoded is dealt with. 

In the present embodiment, the page load unit 206 loads 
the contents of the designated time line stream from the 55 
CD-ROM 201 to the page storage unit 210 at a time, when 
the execution" of the time line information is started: the 
display data creation unit 213 reads the designated still 
picture data in the loaded time line stream and creates 
display data. However, the contents of the time line stream 60 
does not necessarily have to be loaded at a time. For 
example, the page load unit 206 can load the designated still 
picture data and audio data in the designated timed line 
stream to the page storage unit 210 at each display time of 
the dynamic picture display, and the display data creation 65 
unit 213 can read the loaded still picture data from the page 
storage unit 210 and create display data, and the time line \ 
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control unit 207 can transfer the loaded audio data to the 
audio data storage unit 211. Generally, the amount of the 
compressed and encoded image data is large, so the data 
amount shown by one time line stream can be considered 

5 rather large. Therefore, considering the storage capacity of 
the RAM 302. which is necessary as the page storage unit 
210. the page load unit 206 had better load the designated 
still picture data and audio data to the page storage unit 210 
at each display time of the dynamic picture display. As the 

10 contents of the time line stream is successively stored on the 
same track, seek of a read head docs not happen, even if the 
designated still picture data and audio data are loaded from 
the CD-ROM 201 at each display time. This enables dis- 
playing of dynamic pictures without breaks in sound and 

15 images. 

In the present embodiment, the time line control unit 207 
instructs the page load unit 206 to load the time line stream 
designated in the time line information in time of dynamic 
picture display: the page load unit 206 instructs the 

20 CD-ROM interface stream unit 202 to read the time line 
stream from the CD-ROM 201. But the time line stream 
does not necessarily have to be read in this way. and the time 
line control unit 207 can direcdy instruct the CD-ROM 
interface unit 202 to read the designated time line stream, 

25 In the present invention, the display unit 212 displays 
each page every Vy> second or so. But in order to display 
dynamic picture images of high quality, each page has to be 
displayed every ■/<» or Vxso second or so. In this case, still 
picture data and audio data corresponding to the display time 

30 of every ! /eo second or ! /i5o second from the start of the 
execution of the time line stream are shown. 

In FIG. 4. the page stream and the time line stream 
correlating with each other, are stored successively in the 

55 storage areas on the track of the CD-ROM 201. The most 
important point is that contents of each page stream or each 
time line stream is stored successively on the same track: the 
contents of the page stream and the time line stream do not 
have to be correlating with each other. For example, only 
one of the page stream and the time line stream can be stored 
successively in the storage areas. 

In the present embodiment, the CD-ROM 201 is 
explained as the recording medium, but other storage 
medium like DVD can substitute for the CD-ROM 201. 

45 Broader interpretation is possible: the recording medium can 
be a transmission line like a communication line, or a 
storage device of a video server mentioned later. In the 
storage area, all information does not have to be stored in the 
form of digital data. Part of the data. e.g. audio data of the 

^ time line stream, can be analog data and can be stored 
divided. In this cased, the multi-media information playback 
device 200 can be equipped with a storage unit of analog 
data instead of the audio data storage unit 211. 
(Embodiment 2) 

55 FIG. 12 is a block diagram of construction of a multi- 
media information record device 1200 of the second 
embodiment of the present invention. FIG. 13 is a block 
diagram of the hardware construction of die multi-media 
information record device 1200 shown in FIG. 12. 

6C As shown in FIG. 12. the multi-media information record 
device 1200 comprises an input unit 1201. a stream creation 
unit 1202. a second storage unit 1203. a CD-ROM write unit 
1209 and a CD-ROM 1210. The second storage unit 1203 
comprises a partial image storage unit 1204. a dynamic 

65 picture storage unit 1205. an audio storage unit 1206. a page 
stream storage unit 1207 and a time line stream storage unit 
120S. 
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As shown in FIG. 13. elements of the hardware construc- 
tion of the multi-media information record device 1200 is 
almost the same as the multi-media information playback 
device 200 shown in FIG. 3. except that the multi-media 
information record device 1200 does not comprise the joy 5 
pad 306 but comprises a hard disc 1310. 

The input unit 1201 is implemented by a key board 1305 
and an input device interface 1307. The input unit 1201 
comprises a scanner and a video capture board, both of 
which are not illustrated. The scanner inputs printed still to 
pictures as digitalized still picture data. The video capture 
board inputs motion pictures from a connected video tape 
recorder as digitalized dynamic picture data. 

The stream creation unit 1202 is implemented by a CPU 
1301. According to the processing procedure, this unit 1202 U 
instructs a CRT 1317 to display a menu screen and an input 
request, and receives input of instruction and character 
information from an editor of multi-media information. 
According to the instruction, the stream creation unit 1202 
reads a partial image file, a dynamic picture file and audio 20 
file corresponding to each instruction, from the partial image 
storage unit 1204. the dynamic picture storage unit 1205. 
and the audio storage unit 1206. respectively, and interac- 
tively creates the page stream and the time line stream 
shown in FIGS. 5 and 6. Input of the control information is 25 
carried out by a character input from the input unit 1201. The 
processing of the stream creation unit 1202 is explained 
later, by means of flow charts in FIGS. 14 and 15. 

The second storage unit 1203 is implemented by a hard 
disc 1310. This unit comprises the partial image storage unit 30 
1204. the motion picture storage unit 1205, the audio storage 
unit 1206. the page stream storage unit 1207. and the time 
line stream storage unit 1208. 

The partial image storage unit 1204 stores one or a 
plurality of partial image file(s). One partial image is already 35 
cut 'out from material data for expressing a still picture. The 
partial image is shown by bit map data or graphic data which 
are compressed and encoded according to MPEG, and stored 
in each partial image file. Contents in each partial image file 
correspond to the partial image data of the page stream. 40 
When the material data is correlated with control informa- 
tion or other material data by editing, it constitutes multi- 
media information. 

The dynamic picture storage unit 1205 stores one or a 
plurality of dynamic picture file(s). In the dynamic picture 45 
file, still picture data cut out from the material data of still 
pictures for expressing dynamic pictures are stored in order 
of the display rime, corresponding to a predetermined dis- 
play time. 

The audio storage unit 1206 stores one or a plurality of 50 I 
audio filers). Audio data divided into the amount of sound I 
data, which corresponds to the length of time in which one 
still picture data is displayed, are stored in order of the 
display time. I 

The page stream storage unit 1207 stores the page stream 
created by the stream creation unit 1202. 

The time line stream storage unit 1208 stores the time line - 
stream created by the stream creation unit 1202. 

The CD-ROM write unit 1209 is implemented by a disc 60 
interface 1308. According to the instruction of the input unit 
1201. this unit 1209 reads the page stream and the time line 
stream from the page stream storage unit 1207 and the time 
line stream storage unit 1208. respectively, and stores them 
in the CD-ROM 1110. 63 

FIGS. 14 and 15 show flow charts showing procedure of j 
storing multi-media information in the multi-media infor- S 
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mation record device 1200 of the present invention. Steps 
1419-1428 are shown in a flow chart in FIG. 15. 

The stream creation unit 1202 initializes the identification 
number of the page stream and the time line steam to *T\ 
5 (Step 1401) 

The stream creation unit 1202 requests an input instruct- 
ing whether contents to be described exist in the page 
stream. When the contents to be described exist in the page 
stream (Step 1402). the stream creation unit 1202 initializes 

10 the partial image ID 4 T (Step 1403). 

The scream creation unit 1202 further requests input 
instructing whether contents to be described as a partial 
image of the partial image ID exist. When the content exist 

15 (Step 1404;. the stream creation unit 1202 receives a name 
of the partial image file which shows the partial images (Step 
1405). Also, the scream creation unit 1202 receives input of 
X-Y coordinates. X-Y enlargement rates and a rotational 
angle, each of which describing the display position. 

20 enlargement rate and rotational angle of the partial image on 
the initial screen of the page stream (Step 1406). The stream 
creation unit 1202 writes the partial image ID. X-Y 
coordinates. X-Y enlargement rates and the rotational angle 
in the storage areas allocated to each item of the partial 

25 image ID of the page stream in the page stream storage unit 
1207. Then the stream creation unit 1202 reads the partial 
image file, whose name is inputted, from the partial image 
storage unit 1204. and writes the contents of the file in the 
storage areas allocated to each item of the partial image data 
having the partial image ID of the page stream in the page 
stream storage unit 1207 (Step 1407). Then, the stream 
creation unit 1202 increments the partial image ID (Step 
1408). and goes back to Step 1404. 
In Step 1404. when the contents to be described as the 

35 partial image of the partial image ID does not exist, the 
stream creation unit 1202 further requests input instructing 
whether the page stream has control information to be 
described. When the page stream has the control information 
(Step 1409). the stream creation unit 1202 receives input of 

40 a name of the input signal (Step 1410). and receives input of 
commands to be executed according to the name of the input 
signal (Step 1411). The stream creation unit 1202 writes the 
name of the inputted input signal and the command in non 
working area of the storage area allocated to the items of the 
. 45 control information of the page stream in the page stream 
storage unit 1207 (Step 1412). and then goes back to Step 
1409. 

In Step 1409. when the contents to be described as the 
partial image of the control information do not exist, the 

50 stream creation unit 1202 further requests input instructing 
whether the page stream has animation display to be 
described. When the animation display exists (Step 1413). 
the stream creation unit 1202 receives input of the partial 
image ID of the partial image to be displayed in animation 

55 (Step 1414). and receives input of one or a plurality of types 
of animation instructing how the partial image of the partial 
image ID should be displayed in animation (Step 1415). The 
types of animation is chosen from "movement", "enlarge**, 
"rotation"', and the like. A parameter, which changes with the 

6C passage of time is designated according to the input of the 
type of animation. The stream creation unit 1202 requests 
input instructing whether a pair of a time and a value, both 
of which should be shown by the page stream, exists; the 
time being expressed by the elapsed time from the start of 

65 the execution of the time Line information and the value 
corresponding to the types of inputted animation, and the 
time mentioned above. When the pair exists (Step 1416). the 
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stream creation unit 1202 receives the input (Step 1417). 
Specifically, as parameters corresponding to the time men- 
tioned above, values of the X-Y coordinates, the X-Y 
enlargement rates, and the rotational angle are inputted for 
"movement". "enlargement", and "rotation", respectively. 5 

In Step 1416. when the pair does not exist, the stream 
creadon 1202 writes the partial image ID of the partial image 
to be displayed in animation and the pair of the time and the 
parameter iD the non working are as of the storage areas 
allocated to the information relating to the animation display 10 
of the page stream in the page scream storage unit 1207 (Step 
1418). and goes back to Step 1413. 

In Step 1413. when the animation display to be described 
does not exist in the page stream, the stream creation unit 
1202 requests input instructing whether dynamic picture 15 
display to be described exists in the page stream. When the 
dynamic picture display to be described exists (Step 1419). 
the stream creation unit 1202 receives input of a start and 
end times of the dynamic picture display, and the partial 
image ID of the partial image which should be displayed in 20 
dynamic picture (dynamic picture display object) (Step 
1420). 

The stream creation unit 1202 further receives input of the 
name of the dynamic picture file which shows the dynamic 
picture to be displayed inside the partial image shown by the 25 
inputted partial image ID (Step 1421). 

Next, the stream creation unit 1202 requests input 
instructing whether there is an audio file which shows sound 
to be played back synchronized with the dynamic picture 
shown by the name of the inputted file. When the audio file 30 
exists (Step 1422). the stream creation unit 1202 receives 
input of the name of the audio file (Step 1423). Next, the 
stream creation unit 1202 reads the dynamic picture file and 
audio file from the dynamic picture storage unit 1205 and the 
audio storage unit 1206. respectively, and writes the still 35 
picture data and a pair of still picture data and audio data of 
the same time in the storage areas allocated to the time line 
stream having the identification number in the time line 
stream storage unit 1208 (Step 1424). 

The stream creation unit 1202 writes the start and end 40 
times of the inputted dynamic picture display, and the partial 
image ID of the dynamic picture display object, along with 
the identification number of the time line stream, in the non 
working areas allocated to the information relating to the 45 
dynamic picture display of the page stream in the page 
stream storage unit 1207 (Step 1425). 

In Step 1422, when the audio file does not exist, the 
stream creation unit 1202 reads the dynamic picture file 
whose name is inputted from the dynamic storage unit 1205. ^ 
and writes the contents of the file in the storage areas 
allocated to the time line stream having the identification 
number in the time line stream storage unit 1208 (Step 
1426). and then starts Step 1425. 

Next, the stream creation unit 1202 increments the iden- 55 
tification number of the time line stream (Step 1427). and 
goes back to Step 1419. 

In Step 1419. when the dynamic picture display to be 
described does not exit in the page stream, the stream 
creation unit 1202 increments the identification number of 6C 
the page stream (Step 1428). and goes back to Step 1402. 

In Step 1402. when the contents to be described do not 
exist in the page stream, the stream creation 1202 requests 
the input instructing order of recording the page stream and 
the time line stream successively onto the storage areas of 65 
the track of the CD-ROM 1210 (Step 1429). The scream 
creation unit 1202 posts the recording order to the CD-ROM 
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write unit 1209. The CD-ROM write unit 1209 reads the 
page stream and the time line stream from the page stream 
storage unit 1207 the time line stream storage unit 1208. 
respectively, and stores them in the CD-ROM 1210 (Step 
5 1430;. 

As mentioned above, one or a plurality of partial images 
can be shown by one page stream, and multi-media can be 
recorded in the CD-ROM 1210: the multi-media information 
in which parameters for moving, enlarging/reducing, rotat- 

10 ing partial images, commands to be executed according to 
the input signal, and the time line information are described 
by the page stream. Also, each page stream and time line 
stream can be stored successively in the storage areas on the 
same track of the CD-ROM 1210. According to the present 

15 embodiment, the multi-media information stored in the 
multi- media information record device 1200 can be played 
back from the CD-ROM 1210 effectively by the multi-media 
information playback device 200. 

According to the present embodiment, the still picture 

20 data and the audio data shown by the time line stream are 
compressed and encoded according to the MPEG, and they 
are extended and coded by the MPEG decoder 311. But. the 
still picture data, the audio data, and the partial image data 
can be compressed and encoded by other formats like VFW 

25 (Video for Windows) or QT (Quick Time). And the MPEG 
decoder 311 can be substituted for a decoder following other 
formats. When the multi-media information record device 
1200 only deals with data which is not compressed and 
encoded, there is no problem without a decoder. 

30 (Embodiment 3) 

FIG. 16 is a block diagram showing construction of the 
multi-media information playback device 1600 of the third 
embodiment of the present invention. FIG. 17 is a block 

35 diagram showing hardware construction of the multi-media 
information playback device shown in FIG. 16. 

As shown in FIG. 16. the multi-media information play- 
back device 1600 is connected to a data server 1700 through 
a network, comprising a network interface unit 1601 and a 

w page load unit 1602. Other elements are the same as the 
multi-media information playback device 200 shown in FIG. 
2. so only the same reference numbers are given without 
explanation. The multi-media information can be equipped 
with the CD-ROM 201 and the CD-ROM interface unit 202. 

45 As shown in FIG. 17. the hardware construction of the 
multi-media information playback device 1600 is the same 
as that of the multi-media information playback device 200 
shown in FIG. 3 except that it comprises a network interface 
1701. Explanation of the hardware construction is omitted. 

50 since it is the same as that of the multi-media information 
playback device 200. so only the same reference numbers 
are given. 

The data server 1700 is an external device comprising 
storage devices like a hardware disc device, which is not 

55 illustrated. The data server 1700 stores the page stream 
shown in FIG. 5 and the time line stream shown in FIG. 6 
in the storage device, one page stream being stored as one 
file and one time line stream as one file. The data server 1700 
reads files which contains the corresponding page stream 

6c and time line stream from the hard disc device according to 
the read request from the multi-media information playback 
device 1600 via the network, and transfers the page stream 
and the time line stream to the multi-media information 
playback device 1600. 

65 The network interface unit 1601 is implemented by a 
network interface 1701. According to the instruction of the 
page load unit 1602. this unit 1601 reads multi-media 
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information described by the page stream and the time line 
stream stored in the data server 1700. Specifically, the 
network interface unit 1601 transmits PT partition showing 
whether the read request information is the page stream 
TP') or the rime line stream CT") as for the page stream or 5 
the time line stream designated by the page load unit 1602. 
along with the identification number and the read request to 
the data server 1700 via the network. The network interface . 
unit 1601 receives the page stream or the time line stream 
transmitted from the data server 1700 according to the read 10 
request via the network. 

The page load unit 1602 is implemented by the CPU 301. 
Processing of the page load unit. 1602 is the same as that of ! 
the page load unit 206 of the first embodiment except that 
the read request of the page stream and the time line stream 15 
are given to the network interface unit 1601 instead of the 
CD-ROM interface unit 202. 

Explanation about the procedure of the playback of the 
multi -media information playback device 1600 follows next, 
only focusing the difference between the multi-media infor- 20 
mation playback devices 1600 and 200. 

In Step 701, when the multi-media information playback 
device 1600 is activated, the page load unit 1602 instructs 
the network interface u nit 160 1 to read a predetermined page 25 
stream, e.g. the page stream 1 shown in HG. 8. from the data 
server 1700. The network interface unit 1601 transfers a PT 
partition "P". an identification number "1" and its read 
request to the data server 1700. and receives the page stream 
"P from the data server 1700. The page load unit 1602 loads ^ 
the page stream to the page storage device 210. 

The following explanation is about Steps 705-707 when 
the page stream 1 is loaded. 

When the identified input unit is B, the control informa- 
tion execution unit 204 executes a command "Start the clock 35 
of the time line." which corresponds to the input signal B. 
Specifically, the control information execution unit 204 
instructs the time line control unit 207 to start the execution 
of the time line information described by the page stream 1. 
Next, the time line control unit 207 activates the clock 205 40 
and reads the time line information in the page steam 1. 
When the designated time line stream exits in the time line 
information which describes the dynamic picture display, 
the time line control unit 207 instructs the page load unit 
1602 to load the time line stream. In the page stream 1 45 
shown in FIG. 8. an identification number of the time line 
stream to be displayed in dynamic picture "1" is designated, 
so the time line control unit 207 instructs the page load unit i 
1602 to load the time line stream 1 to the page storage unit 
210. The page load unit 1602 instructs the network interface 5c 
unit 1601 to read the time line stream 1. and the network 
interface unit 1601 transfers a PT partition *T*\ an identi- 
fication number "1** and its read request to the data server 
1700. The time line stream 1 from the data server 1700 is 
loaded to the page storage unit 210. The processing after- 55 
wards is the same as that of Embodiment 1. 

When the identified input signal is C. the control infor- 
mation execution unit 204 executes a command "Read the 
page stream 2." which corresponds to the input signal C. 
Specifically, the control information execution unit 204 60 
instructs the page load unit 1602 to load the page stream 2 
to the page storage unit 210. The page load unit 1602 
instructs the network interface unit 1601 to read the page 
stream 2. The network interface unit 1601 transfers a PT 
partition T\ an identification number "2" and its read 65 
request to the data server 1700. The page load unit 1602 
loads the page stream 2 . which is transferred to the network 
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interface unit 1601 from the data server 1700. to the page 
storage unit 210. The processing afterwards is the same as 
that of Embodiment 1. 

As mentioned above, the multi-media information play- 

5 back device 1600 produces the same effect as that of the 
multi-media information playback device 200 of Embodi- 
ment 1 by the page stream and the time line stream from the 
data server 1700. according to the present embodiment. 
The data server 1700 does not necessarily have to store 

10 the page stream and the time line stream, separately. The 
data server 1700 can store them by the identification num- 
bers of each file expressed by ordering numbers and the like. 
In this case, designation of the page stream and the time line 
stream described by the page stream information and the 

15 time line stream information, and the designation of the page 
stream and the time line stream by the parameters of the 
control information execution unit 204 and the time line 
control unit 207 are carried out by the identification numbers 
of each file. 

20 In the present invention, the contents of the time line 
stream do not necessarily have to be loaded to the page 
storage unit 210 at a time. For example, the page load unit 
206 can load the designated still picture data and the audio 
data in the designated time line stream to the page storage 

25 unit 210 at every display time of the dynamic picture 
display: the display data creation unit 213 can read the 
loaded still picture data from the page storage unit 210 and 
create display data: the time line control unit 207 can 
transfer the loaded audio data to the audio data storage unit 

30 211. The following processing should be carried out in order 
to realize the variations mentioned above. The network 
interface unit 1601 and the data server 1700 reserve the 
communication areas of the network for a given time 
corresponding to the dynamic picture display execution 

35 time. Then the data server 1700 reads the still picture data 
and the audio data when they are compressed and encoded 
according to the MPEG and the like from the file which 
records time line stream of the read request. The page load 
unit 1602 loads the still picture image data and the audio 

40 data to the page storage unit 210 in the multi-media infor- 
mation playback de%ice 1600 which receives the still picture 
data and the audio data corresponding to each display time 
via the network. The loaded still picture data and the audio 
data are played back at the proper display time by each of the 

45 elements of the multi-media information playback device 
1600. In the data server 1700. one time line stream is stored 
in one file. So when the still picture data and the audio data 
in the same time line stream are read, read request to the data 
server 1700 does not happen. So. according to the present 

50 invention, even though the contents of the time line stream 
is not loaded to the page storage unit 210 at a time, the still 
picture data and audio data corresponding to the display time 
from the same time line stream are read at high speed. So 
breaks does not happen in the sound and dynamic pictures. 

55 When the dynamic picture display, the time line control 
unit 207 can directly instruct the network interface unit 1601 
to read the designated time line stream. 
(Embodiment 4) 

60 FIG. 18 is a block diagram showing the construction of 
the multi-media information record device 1800 of the 
fourth embodiment of the present embodiment FIG. 19 is a 
block diagram showing hardware construction of the multi- 
media information record device 1800 shown in FIG. 18. 

65 As shown in FIG. 18. the multi-media information record 
device 1800 is connected to the data server 1700. instead of 
the CD-ROM write unit 1209 and the CD-ROM 1210 of the 
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multi-media information record device 1200 shown in FIG. 
12. via a communication line, and comprises a data server 
write unit 1801. Other elements are the same as the multi- 
media information record device 1200. so explanation is 
omitted. Only reference numbers are given. 5 

As shown in FIG. 19. the hardware construction of the 
multi-media information record device 1800 comprises a net 
interface 1901. instead of the CD-ROM 1309 of the multi- 
media information record device 1200 shown in FIG. 13. 
Other elements are the same as the multi-media information 10 
record device 1200 shown in FIG. 13. So explanation is 
omitted. Only reference numbers are given. 

The data server 1700 receives the page stream and the 
time line stream transferred from the data server write unit 
1801 along with a write request via a communication line. 15 
and stores them in the storage devices inside, e.g. a hard disc 
device which is not illustrated, as individual file. 

The data server write unit 1801 is implemented by the 
network interface 1901. The processing of the data server 
write unit 1801 is the same as that of the CD-ROM write unit 
1209 shown in FIG. 12. but the created page stream and the 
time line stream are stored in the storage device inside the 
data server 1700 which is connected via a communication 
line, instead of the CD-ROM 1210. The data server write 
unit 1801 reads the time line stream from the time line 
stream storage unit 1208 according to the notification from 
the stream creation unit 1202. and transfers a FT partition 
*T* of the time line stream, an identification number, and the 
time line stream to the data server 1700 along with the write 
request. Then the data server 1700 records the page stream 
in the file having the PT partition "P" and the identification 
number, and records the time line stream in the file having 
the FT partition *T* and the identification number. 

According to the present embodiment, the multi-media 33 
information record device 1800 can stored the page stream 
and the time line stream shown in FIG. 5 and FIG. 6 in the 
storage device inside the data server 1700 as one file. The 
present embodiment can produce the same effect mentioned 
in Embodiment 1 by playing back the page stream and the ^ 
rime line stream recorded, by the multi-media information 
record device 1800. in the multi-media informadon play- 
back device 1600 connected to the data server 1700 via a 
network. The data server 1700 can store the page stream and 
the time line stream by the identification numbers of each 43 
file, without distinguishing the page stream and the time line 
stream. In this case, the designation of the page stream and 
the time line stream described in the information in the page 
stream and the time line stream, and the designation of the 
page stream and the time line stream by the parameters of ^ 
the control information execution unit 204 and the time line 
control unit 207. are carried out by the identification num- 
bers of each of the files. 

Although the present invendon has been fully described 
by way of examples with reference to the accompanying 35 
drawings, it is to be noted that various changes and modi- 
fications will be apparent to those skilled in the art 
Therefore, unless such changes and modifications depart - 
form the scope of the present invention, they should be 
construed as being included therein. $c 



